Bias in Deep Learning and Applications to Face Analysis

Deep learning has fostered the progress in the field of face analysis, resulting in the integration of these models in multiple aspects of society. Even though the majority of research has focused on optimizing standard evaluation metrics, recent work has exposed the bias of such algorithms as well as the dangers of their unaccountable utilization.n this thesis, we explore the bias of deep learning models in the discriminative and the generative setting. We begin by investigating the bias of face analysis models with regards to different demographics. To this end, we collect KANFace, a large-scale video and image dataset of faces captured ``in-the-wild’'. The rich set of annotations allows us to expose the demographic bias of deep learning models, which we mitigate by utilizing adversarial learning to debias the deep representations. Furthermore, we explore neural augmentation as a strategy towards training fair classifiers. We propose a style-based multi-attribute transfer framework that is able to synthesize photo-realistic faces of the underrepresented demographics. This is achieved by introducing a multi-attribute extension to Adaptive Instance Normalisation that captures the multiplicative interactions between the representations of different attributes. Focusing on bias in gender recognition, we showcase the efficacy of the framework in training classifiers that are more fair compared to generative and fairness-aware methods.In the second part, we focus on bias in deep generative models. In particular, we start by studying the generalization of generative models on images of unseen attribute combinations. To this end, we extend the conditional Variational Autoencoder by introducing a multilinear conditioning framework. The proposed method is able to synthesize unseen attribute combinations by modeling the multiplicative interactions between the attributes. Lastly, in order to control protected attributes, we investigate controlled image generation without training on a labelled dataset. We leverage pre-trained Generative Adversarial Networks that are trained in an unsupervised fashion and exploit the clustering that occurs in the representation space of intermediate layers of the generator. We show that these clusters capture semantic attribute information and condition image synthesis on the cluster assignment using Implicit Maximum Likelihood Estimation.Open Acces

Learning Neural Parametric Head Models

We propose a novel 3D morphable model for complete human heads based on hybrid neural fields. At the core of our model lies a neural parametric representation that disentangles identity and expressions in disjoint latent spaces. To this end, we capture a person's identity in a canonical space as a signed distance field (SDF), and model facial expressions with a neural deformation field. In addition, our representation achieves high-fidelity local detail by introducing an ensemble of local fields centered around facial anchor points. To facilitate generalization, we train our model on a newly-captured dataset of over 3700 head scans from 203 different identities using a custom high-end 3D scanning setup. Our dataset significantly exceeds comparable existing datasets, both with respect to quality and completeness of geometry, averaging around 3.5M mesh faces per scan 1 1 We will publicly release our dataset along with a public benchmark for both neural head avatar construction as well as an evaluation on a hidden test-set for inference-time fitting.. Finally, we demonstrate that our approach outperforms state-of-the-art methods in terms of fitting error and reconstruction quality

HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion

Representing human performance at high-fidelity is an essential building block in diverse applications, such as film production, computer games or videoconferencing. To close the gap to production-level quality, we introduce HumanRF, a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints. Our novel representation acts as a dynamic video encoding that captures fine details at high compression rates by factorizing space-time into a temporal matrix-vector decomposition. This allows us to obtain temporally coherent reconstructions of human actors for long sequences, while representing high-resolution details even in the context of challenging motion. While most research focuses on synthesizing at resolutions of 4MP or lower, we address the challenge of operating at 12MP. To this end, we introduce ActorsHQ, a novel multi-view dataset that provides 12MP footage from 160 cameras for 16 sequences with high-fidelity, per-frame mesh reconstructions. We demonstrate challenges that emerge from using such high-resolution data and show that our newly introduced HumanRF effectively leverages this data, making a significant step towards production-level quality novel view synthesis.Comment: Project webpage: https://synthesiaresearch.github.io/humanrf Dataset webpage: https://www.actors-hq.com/ Video: https://www.youtube.com/watch?v=OTnhiLLE7io Code: https://github.com/synthesiaresearch/humanr

Neural Network-Enabled Identification of Weak Inspiratory Efforts during Pressure Support Ventilation Using Ventilator Waveforms

During pressure support ventilation (PSV), excessive assist results in weak inspiratory efforts and promotes diaphragm atrophy and delayed weaning. The aim of this study was to develop a classifier using a neural network to identify weak inspiratory efforts during PSV, based on the ventilator waveforms. Recordings of flow, airway, esophageal and gastric pressures from critically ill patients were used to create an annotated dataset, using data from 37 patients at 2–5 different levels of support, computing the inspiratory time and effort for every breath. The complete dataset was randomly split, and data from 22 patients (45,650 breaths) were used to develop the model. Using a One-Dimensional Convolutional Neural Network, a predictive model was developed to characterize the inspiratory effort of each breath as weak or not, using a threshold of 50 cmH2O*s/min. The following results were produced by implementing the model on data from 15 different patients (31,343 breaths). The model predicted weak inspiratory efforts with a sensitivity of 88%, specificity of 72%, positive predictive value of 40%, and negative predictive value of 96%. These results provide a ‘proof-of-concept’ for the ability of such a neural-network based predictive model to facilitate the implementation of personalized assisted ventilation

Humidity Sensing Properties of Paper Substrates and Their Passivation with ZnO Nanoparticles for Sensor Applications

In this paper, we investigated the effect of humidity on paper substrates and propose a simple and low-cost method for their passivation using ZnO nanoparticles. To this end, we built paper-based microdevices based on an interdigitated electrode (IDE) configuration by means of a mask-less laser patterning method on simple commercial printing papers. Initial resistive measurements indicate that a paper substrate with a porous surface can be used as a cost-effective, sensitive and disposable humidity sensor in the 20% to 70% relative humidity (RH) range. Successive spin-coated layers of ZnO nanoparticles then, control the effect of humidity. Using this approach, the sensors become passive to relative humidity changes, paving the way to the development of ZnO-based gas sensors on paper substrates insensitive to humidity